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Abstract: Technology assessment is a systematic examination of the effects on or of new 
developments such as technologies, processes, policies, organizations, and so on. In this paper, we 
present a systematic method for technology assessment, as a part of the suite of tools for Forecasting 
Innovation Pathways (FIP). We explore means to combine tech mining tools with human intelligence 
in several idea exchange rounds to uncover potential secondary effects, and array them in terms of 
likelihood and magnitude. Big Data is studied as the case study. This is on-going research. We are 
currently on the 2" round of stage 2. Technology assessment is a necessary component of FIP. It 
identifies areas in which significant impacts may occur, their likelihood, and their significance. The 
forecaster must evaluate these impacts, consider measures to enhance or inhibit them, and factor 
them into the planning process for developing the technology. 


I. Introduction 


In recent years, we have developed a suite of tools for Forecasting Innovation Pathways (FIP). FIP 
builds on “tech mining” [1,2], especially of global database search results on the technology under 
study. Robinson et al.[3] laid out four stages and ten steps. The third stage of the FIP approach 
includes “Technology Assessment.” 


Technology Assessment (TA) has dual meanings. For one, it concerns the evaluation of alternative 
technologies, i.e., comparing current or evolving technologies in terms of specific objectives. But, TA 
also refers to a second, distinct set of activities — namely, “impact assessment” — i.e., studying the 
future broad, societal effects of the development and application of emerging technologies [4]. A 
classic definition directs attention to the “unintended, indirect, and delayed” effects of such 
development [5]. That is the focus of this paper. 


In FIP development to date, our TA efforts have received less attention; in this paper we address 
impact identification and assessment. We develop and apply essential impact assessment aids to 
identify high likelihood and/or high magnitude effects associated with developmental choices. The 
paper aims for a systematic process for TA that identifies potential slow emerging impacts, quickly 
and efficiently: 


1) Address the full framework, including foundations, uses (applications), as well as the 
potential impacts (both positive and negative) 
2) Experiment with tech mining tools to elicit indications of potential secondary effects by 
developing impact taxonomies 
3) Perform basic analyses of the potential effects identified to array them in terms of likelihood 
and magnitude -- then focus attention on impacts that appear either high likelihood and/or 
high magnitude 
4) Present those results for review and for exploration of candidate mitigation measures to 
treat undesirable impacts — both via traditional workshops and via internet-enabled modes; 
compare those. 
“Big Data” is the focus for this empirical case analysis. Big Data portends momentous implications 
for multiple sectors, offering a timely and rich topic for exploration of potential impacts. In this paper, 
Section 2 explains the contextual framework & research approach. Big Data is illustrated as a case in 
Section 3. Section 4 discusses implications. 


II. Contextual Framework & Research Approach 


Technology assessment is a meta-level method used to analyze potential development pathways of a 
technology and the social and economic implications of this development[6,7]. Included in 
technology assessment are methods to perform empirical analysis of the emerging technology; 
methods to engage stakeholders, experts, and publics, and methods to assess future pathways [8]. 
Technology assessment does not presume to provide accurate predictions of the future. Rather it 
seeks to reduce the uncertainties that restrict investment in the technology through revealing and, 
presumably, encouraging attention to negative societal impacts [9,10]. Technology assessment has 
traditionally been a central government function [the US Congress as of 1995 no longer has an Office 
of Technology Assessment to study the likely impacts of new technologies, but other US organizations 
are involved in technology assessment (or quasi-technology assessment)] including the National 
Academies and the General Accountability Office (GAO). However, decentralized methods have arisen 
to obtain more diverse inputs as the technologies are emerging [11,12]. 


Beginning Spring, 2015, with U.S. National Science Foundation (NSF) support, we have been working 
on “Forecasting Innovation Pathways of Big Data & Analytics“. Our research has two main elements: 1) 
‘tech mining’ (empirical analyses of research funding, literature, and patents to discern R&D trends 
and active players), and 2) engagement of stakeholders and experts to help understand 
developmental prospects and likely outcomes. The paper aims toward a systematic process for 
impact identification, analysis, and evaluation. We seek first to identify potential impacts, quickly and 
efficiently: (The process is not linear; “effects of effects” could be important; enhanced exploration of 
innovation pathways should uncover additional effects.) 


Fig. 1 shows our Contextual Framework for Technology Assessment 
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Figure 1. Contextual Framework for Technology Assessment 


Stage 1: Identifying impacts, we aim to address the full developmental pathways — i.e., consider 
implications of the processes, as well as the resulting applications. We do so under scrutiny by: 1) 
bibliometric methods. Search strategies are needed here. The database could be Web of Science, etc. 
Here, we choose ProQuest Business database, which may reflect consideration of impacts in 
conjunction with developing applications. 2) Literature review is also important here. As for some 
emerging technologies, impact discussion can be very informative in reviews, foresight studies, 
consulting reports, etc. 3) Use of a web crawler might also help, but we have not utilized that in this 
study. After scanning to identify the impacts, we separate the candidate impacts using select criteria 
— i.e., by positive vs. negative; affecting organizations/individuals/society; by particular application 
arenas, data driven vs. problem driven, etc. Also, we sort the potential effects identified to array 
them for consideration of their likelihood and magnitude. 


Stage 2: We strive to present those results in concise, easily digested format for review by persons 
whom we have identified through our “tech mining” of databases and review of key papers and for 
whom we have obtained e-mail addresses. We ask them to improve our set of potential impacts of 
interest. Then, we digest what we hear to summarize Big Data “impact identification.” Moreover, 
we seek open internet inputs to enrich that characterization, clarify preferences of various 
stakeholders, and posit policy actions warranting attention. 


Stage 3: We pursue research to explore the impacts, seeking data to support estimations of likelihood 
and magnitude, identify contingencies and dependencies, identify stakeholder perspectives, etc. 


Policy suggestions should be the final outcomes for this Stage. 
III. Illustrative case: Big Data & Analytics 


Although the legacy of information technology development is long, the term “Big Data” has a more 
recent history. Some trace the notion of Big Data to a special issue of Nature published in September, 
2008, on the topic, while others allude to earlier or later references. Indeed the term itself has 
become a “meme” for developments in the 21* century that facilitate the procurement, storage, 
processing, and analysis of large-scale information compilations. Boyd and Crawford [13] call out the 
“mythology” of the term, associating it with an overly optimistic and opportunistic rhetoric. The 
White House[14] has drawn on the Gartner, Inc. definition of Big Data in terms of the three “Vs” 
(although more V’s have been added in other definitions): (1) volume of data collected and processed 
at a decreasing cost; (2) variety of data, including digital data and data originating in analog forms 
that can be digitized (see President’s Council of Advisors on Science and Technology, 2014); and (3) 
velocity of data that can be obtained nearly in real-time. The ability to process more information, 
more quickly, and with greater ease of analysis opens up opportunities in medical, business, scientific 
research, environmental, defense, and climate change applications, among others [15]. Concerned by 
the great potential, but also imposing impacts, GAO has initiated a “21st Century Data” TA in 2015. 
This is being undertaken on behalf of the Comptroller General (i.e., at GAO initiative). This presents 
an intriguing opportunity for our Georgia Tech based team to address this, not uncommon, gap 
between historical and future-oriented analyses. | We propose to experiment with our Forecast 
Innovation Pathways methodology on this “big data & analytics” Assessment. In doing so, we will 
be in position to present how FIP empirical methodology can provide useful insights into innovation 
prospects and implications of Big Data, the “internet of things,” quality and privacy issues, and such. 


A. Understand the technology & Scan the potential impacts 

Basically, we do tech mining of R&D on “Big Data & Analytics” (BDA) drawn from multiple databases: 
Web of Science (WoS), INSPEC, ABI Inform, NSF and NSFC (National Natural Science Foundation of 
China) awards, and Derwent Innovation Index patents. The analyses show amazing growth in R&D 
and attention to BDA building hyper-exponentially from 2008, but showing indications of saturating 
as of 2015. 


A map of the publications indexed by WoS shows incredibly broad interest — extending way beyond 
computer & data science — in using BDA to advance research in diverse fields. Our analyses find the 
U.S. and China leading the global BDA effort. Here we are emphasizing search results on Big Data 
from the ProQuest Business. database for 2010-2014. We use the terms -- Problem/ risk/ 
challenge/ impact/ effect/ burden/ benefit -- to reduce the 9696 Big Data records to 620 that appear 
to consider impacts. We review topical term lists and read selected abstracts to bolster our 
candidate Big Data impacts set. Besides, we read more than 60 selected articles to widen and 
deepen our understanding of potential impacts of the development, application, and uses of Big Data. 
We have identified some 20 major application areas, pursuing in-depth analyses of select ones (e.g., 
Electronic Health Records -- EHR). Here we aspire to address impacts arising from any of the 
applications to help inform policy considerations. 


We model Big Data applications using a simple 3-level framework (Fig. 2): 
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1) Information Technology (IT) Foundations — Communications, Storage, and Computing that 
enable Big Data functionality [Note: this level is NOT our focus here.] 

2) Analytics — Building on the Big Data foundations — enhanced and new functionality is coming 
into use. We perceive four major functional elements: monitoring; modeling & prediction; 
auditing & evaluation; and process optimization. 

3) Uses, especially oriented to decision support — including four prominent forms: service 
improvement; management & organization; process improvement; and R&D. And then, we 
separate the candidate impacts on select criteria — i.e., positive vs. negative; affecting 
organizations/individuals/society; by particular application arenas, data driven vs. problem 
driven, etc. 

Our focus is on the U.S., with an eye toward potential Federal policy actions to promote beneficial 
development while reducing potential risks and costs. Figure 2 and Table 1 explain the details. Table 1 
tracks how functions (Column 1) could be operationalized (Column 2), leading to Uses (Column 3). 
Column 4 adds example impacts. 
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Figure 2. 3-level Framework for Big Data (this figure should be replaced later.) 


As mentioned, we undertake analyses at two levels for Big Data. At a general level we seek to 
identify notable systemic characteristics and effects of Big Data for many applications. The target is to 
identify potential impacts not limited to a particular application domain. Second, for specific 
applications — focus on one application domain and perform a similar TA process (The EHR impact 
table is not shown here. ) 


2. Exchange ideas & renew the impacts 

1% round 

We found about 50 persons whom we have identified through tech mining of databases and review 
of key papers, and for whom we have obtained e-mail addresses. In the emails we sent to them, we 
laid out what we are pursuing for this research and pointed them to read the project webpage 
(http://bigdatagt.org). We sent them Figure 2 and Table 1 to illustrate how we understand Big Data, 
and welcome their refinements to any of that content, but expressly seek the suggestions regarding 
potential indirect, delayed, or unintended effects (Column 4 of the Table 1). The grey blank cells in 
Table 1 invite any additional points they might offer. We also welcomed ideas on possible policy and 
mitigation options to deter undesirable effects. We sent a reminder 10 days after the 1% round 


| “ 


emails. We digested what we hear back to update the Table 1 to summarize our initial “impact 


identification.” 


Unfortunately, response rate has been low (~12%). This is consistent with other email surveying that 
our Center has done recently. We will invite the 50 researchers to consider our budding website 
that we are developing to share early findings. The weak response prods us to pursue alternative, 
internet-based methods to engage various parties at interest. We are pursuing a blog model, 
providing easy means to stay in touch with interested parties and attract their feedback. [We will 
update results at PICMET.] 


2"4 round 

As noted, we are making a blog to seek open internet inputs to enrich our characterization of Big 
Data implications, clarify preferences of various stakeholders, and posit policy actions warranting 
attention. We’d like to get 2 types of responses — invited (by us) and open (anyone). We invite 
the 50 reviewers of the 1st round to engage the website. We share detailed project descriptions 
and the bibliometric analysis results there, as well as the revised impact table (Table 1 in this paper) 
according to reviewers’ comments. The website is also open to public. Our plan is to track inputs 
from various invitees vs. those from open comment to check for discrepant estimates and valuations. 
More importantly, we aim to understand the likelihood and magnitude of each impact. We have 
drawn sets of major impacts (Table 2) from Table 1, and defined each impact briefly. We’d like to 
have 3 responses for each impact: 1) % (likelihood); 2) Importance (magnitude); 3) Policy action 
suggestions. We ask a likelihood of occurrence in the Year 2026 (10 years from now on). As a 
contingent question, we also ask if that impact does occur, how large an effect will it exert on the US. 
For example: 

e Q1: One possible outcome of widespread BDA application could be displacement of 
“analysts” in many sectors. How likely do you think that BDA will significantly displace 
American analysts by the Year 2026? [0-10] 

e Q2: If, indeed, such displacement occurs, out of some X million analysts working today, what 
would be the net reduction due to BDA as of 2026? [in millions; give the option of an 
increase instead of a decrease too] [0-10] 

e Q3: Any policy action suggestions under such situation? 


We put up a model 2x2 matrix — BDA Impact 
Estimation- to show the likelihood & magnitude survey 
result about potential BDA Effects. This axis shows O-10 
scaling on Likelihood (vertical axis), and O-1O0 on Magnitude 
(horizontal axis) of each effects. Grid origin of the axis is (5, 


5). 


Drawing on literature, discussion, and knowledgeable feedback, we have identified 20 candidate 
impacts (i.e., outcomes or effects) of widening uses of Big Data Analytics over the coming 10 
years. We want to locate these on a 4-quadrant chart to help see the most important effects that 
warrant possible policy actions to encourage or reduce. Fig 3 illustrates several effects in this chart to 
stimulate the likelihood and magnitude. 


BDA Impact Estimation 
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Figure3 BDA Impact Estimation Illustration 
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Table 1. ‘Big Data Analytics’ Effects at Different Levels 


<+ Tracking individual 
behaviors for personalized 
services. 

~*~  Geo-tracking to expedite 
services; 

~ Supply chain management 

(aided by comprehensive, 

real-time analytics to 

dynamically adjust) 

Environmental sensors 


Provide data that can lead to new solutions to major challenges 

Challenge privacy norms (but must distinguish between consensual and 
non-consensual monitoring — e.g., auto insurance plug-ins) 

Secondary use of collected data (who pays, who benefits? Who controls?) 
Politicize; volatile stakeholder attitudes, subject to media manipulation 
Smarter shopping for goods, colleges, health care, etc. 

Potential to reduce moral hazard (real time monitoring may create positive 
changes in behavior) 

More accurate risk pricing as can be based on actual behavior rather than 
correlated attributes or outcomes 


Analytics- Operational | Explanations & Example Uses | Example “Effects of effects” 
based Level [potential “indirect, delayed, or unintended” impacts of Big Data Analytics & Uses, 
Functionality both good & bad] 
Monitoring Data Sharing data among different ~ Foster accountability; 
transparency | sectors, organizations and even <+ Enable expanded network cooperation; 
countries -- enabling <% Pose privacy & security threats 
organizations & individuals to <+ Enable earlier detection of events 
affirm accuracy & completeness; | % Pave the way for further correlation 
Open government Data (OGD) 
Cross-data Combining multiple sources to <+ Gain in analytics due to combinations of diverse data; 
integration enable new forms of inquiry, new | ~ Pose extreme privacy & security threats (even when no single source of data 
analytics (e.g., in agriculture -- reveals identity, correlation across multiple sources can) 
soil, weather, chemical <+ Reduce world hunger (via increased agricultural productivity) 
monitoring to boost agricultural | <> In general, solve major world problems 
yields); ~ Induce jurisdictional fights (“power” in data control) 
requiring organizational ~ Require data reformatting 
coordination ~ able to connect a patient’s complete medical history with prescription drug and 
treatment options; 
V Lead to potential mis-understandings if semantics are not identical or understood 
Remote & Collecting information on <+ Open novel industrial organization opportunities; 
real-time country, organization and ~ Heighten spying concerns; 
monitoring individual behaviors <+ Enable behavior modification, for better or worse, e.g., for better health 
+> 
+> 
+> 
+> 
> 
+> 
+> 
+> 


+ + 


Health sensors 


Can reduce adverse selection, as agreement to be monitored serves as a credible 
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signal of lower risk type. 


<+ give education officials the tools they need to continuously improve the 
educational experience of their students 
Monitoring | Collecting online social media <% Raise ‘Big Brother’ concerns 
public and physical public spaces ~ Increase sense of security 
activities surveillance -- enabling network | <> Reduce terrorism and crime in general 
analyses, enhanced security, <% Negative impacts of surveillance-driven behavioral changes? 
crime control, etc. 
Modeling & | Advanced Using statistical tools and <v Initiate new R&D opportunities; 
Prediction analytics Artificial Intelligence to generate |< Save energy 
evidence-based interpretations ~ Better diagnostics — for health, for industrial systems, etc. 
<+ Protect individuals and businesses by, e.g., predicting extreme weather events, 
crime, ... 
<% Economic benefits from increased efficiency due to analysis 
< manage the most efficient transportation patterns 
Predictive Opportunities to model for ~ Reduce human managerial needs through semi-automated decision support; 
modeling different purposes, such as global | ~~ Facilitate automation (e.g., self-driving cars) 
warming or epidemiological < Enable “expert” help in regions or situations where there is no expertise — e.g., 
prediction (Google Flu Trends); AIDS treatment in poorer regions with the quality of experts, etc. 
natural disaster prediction, <+ More granular and accurate predictions can lead to more efficient pricing and 
market demand prediction, etc. matching > promote separation over pooling equilibria, which increase welfare 
Opportunities for predictions by making the market larger. 
around credit, insurance, and < Have to be careful to distinguish privacy demands that stem from strategic 
labor markets rationales vs. intrinsic demands for privacy. 
m Privacy concerns stemming from one’s true type being revealed (e.g., high 
risk driver or unproductive worker) are strategic. 
mM Privacy concerns stemming from analytics predicting something sensitive, 
and potentially embarrassing (e.g., sexual preference ) are intrinsic. 
E Some privacy demands are mixed (e.g., sensitive health conditions). 
There are strategic reasons for wanting, e.g., drug addiction or depression, 
concealed, but revelation also violates intrinsic privacy demands. 
<% Want to discourage resources used on analytics to effect distribution rather than 


production. 
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E E.g., Using big data to predict a counter party’s willingness to pay merely 
to get a larger share of surplus is dissipative if the transaction would have 
taken place regardless; expending resources to effect a transfer is wasteful. 


<V But, using analytics to predict willingness to pay so that offers can be extended to 
those who otherwise would be left out of the market is efficient, as it increases 
surplus. 
<% track anonymous cell phone user data to quickly identify accidents and other 
traffic challenges 
<+ apply weather models to residential population databases to quickly alert affected 
people 
Virtual Processing vast data resources <% Change the nature of gaming, sports, design 
Reality with real-time speed <+ Increase the number of couch potatoes 
<% Enable better medicine 
< Increase isolation of individuals by reducing face to face human interaction 
Reporting Allow the linkage of multiple 
tools data sets as if you were 
reporting from one data source. 
Auditing & Audit Detection of misuse of funds, <+ Increase accountability; 
Evaluation capabilities; | fraud, and abuses of power; ~ Provide richer input and output intelligence; 
Regulatory Improved customer experiences <% Automate various functions (e.g., taxing) 
& through loyalty programs and <+ Reduce human oversight and understanding 
Compliance | such 
Process Semi-autom | Faster emergency response; <+ Change organizational structures 
Optimization | ated decision | Improving workflow re-design <% Alter human resource allocation (may lead to new departments and new jobs, but 
making (enhance organization loss of others); 
effectiveness) <+ Biases and assumptions inbuilt (consciously or not) into decision-making 
algorithms 
<% Possible errors/oversights from imperfect learning/rules 
< Reduced attention from humans 
Dynamic Based on real-time monitoring, <% Enhance managerial effectiveness; 
plan people could manage various ~ Reduce cost and time; 
management | plans dynamically; <% Reduce managerial and analyst labor needs 
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multi-organizational production 
systems; logistics 


Personalizin | Promoting personalized services | ~ Enhance service effectiveness; 
g services such as personalized medicine; <+ Improve health (and other sector functions) 
model Customer 360 understanding of <+ — Increased consumption 
needs & tailoring of services <% provide more personalized or individualized care for a patient’s specific case 
Targeted advertising 
Enabling Analyzing “natural experiments” | <> Lead to accelerating science, technology & innovation 
real-world, (comparative data), such as <+ Transform to smart cities 
real-time probing comprehensive patient <4 Potential for abuse 
experimentat | and outcome data to compare the | % Improve social & welfare services 
ion effectiveness of various <+ Improve national security & public safety 
interventions <% Save a significant number of lives 
Cumulative Economic Power shifts; ~ Increase or decrease economic equality (“Data Divide”)? 
gains or who owns what data? <+ Favored few gain (e.g., GE targeting the Internet of Things) 
costs Ubiquitousness of data < Generate possible political backlash, leading to unrest & extremism 
Internet of collection; ~ Reasons to belief that poor may gain from big data: 
Things (IoT) | Automated analyses of Big Data o Rich already have access to credit. But, in many cases little information 
combinations on poor, so they are pooled with others in similar circumstances despite 
true ability to pay back. Big data using alternative scoring factors can 
detect most creditworthy within a pool of high-risk borrowers. Empirical 
evidence of credit scoring supports the notion that poor will gain the most 
as they have been excluded from markets. 
o Big data used for price discrimination means that lower prices can be 
targeted at poor. 
~ Big data used in hiring could obviate the need to get a four-year degree to signal 
abilities. This will open the door to poor to get jobs that only college educated 
could obtain before, which could decrease income equality (which is driven 
primarily by returns to education). 
< Strengthen collaboration among countries 
<% Extending new market development from enhancing customer experience 
<% Foreign policy hazards (e.g., Snowdon release of intelligence data) 
< Extensive displacement of human white collar workers 
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Table 2. Major Potential Impacts of Big Data Analytics 


Potential Negative Impacts of Big Data Analytics 

1 Abuse of privacy 

2 Security threats, especially to organizations 

3 Misinterpretations [due to unintended (secondary) use of data compiled for other purposes] 

4 False confidence in predictive power based on historical data [presuming trend stability; 
lacking consideration of events and other forces apt to alter developmental trajectories] 

5 Unquestioning acceptance of conclusions by algorithms or artificial intelligence (Al) — failure 
to examine “black box” assumptions and possible biases or flawed rules 

6 Political backlash against empirical “power usurpation” — leading to ideological anti-data 
unrest and extremism 

7 Human isolation — as individual roles in decision processes reduce, face-to-face interaction 
will possibly diminish. 

8 Jurisdictional fights among organizations [reflecting the increasing “power” resident in 
control of the data] 

9 ?? [we welcome your additions] 

Potential Positive Impacts of Big Data Analytics (BDA) 

10 Data sharing across boundaries (national, organizational) expands networking (cooperation) 

11 Effective monitoring — earlier detection of threats to protect organizations and individuals 
(e.g., terrorism, environmental hazards) 

12 Better predictions [drawing on enhanced data together with increasingly powerful 
computing and effective algorithms] (e.g., weather, crime) 

13 Improved understanding of major challenges, leading to better solutions (e.g., global climate 
change) 

14 Better decisions by individuals and organizations, informed by richer data (e.g., smarter 
shopping, health care, and education choices) 

15 Cumulative benefits of BDA to lower crime and terrorism widely 

16 Economic gain (more granular and accurate modeling and prediction to tune pricing and 
needs-matching) 

17 Enlarged markets — better informed consumers extend their options via more data and 
better analyses 

18 New sectors; new jobs (enhanced resource utilization could enable vast new opportunities) 

19 ?? [we welcome your additions] 


Possibly Positive and/or Negative Impacts of Big Data Analytics 


20 Reduced analyst and manager needs due to semi-automated decision making (reduced cost; 
faster; increased output, but job displacement too) 

21 Economic (in)equality — likely redistribution of wealth [“data divide” could skew toward the 
already advantaged; conversely, more reliance on common assets (especially shared data 
and algorithms) could enable greater equality] 

22 Behavior modification (e.g., better informed lifestyle choices resulting in better health; but, 
potential dumbing down of human roles, leading to passivity or such 

23 ?? [we welcome your additions] 
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IV. Discussion 


Technology assessment is a systematic examination of the effects on or of new developments such 
as technologies, processes, policies, organizations, and so on. Impact assessments are classified as 
policy studies, since they can affect the policies of the organizations that conduct them, as well as 
those of other stakeholders. In most cases, impact assessments should result in actions. Assessments 
may be freestanding or part of another study such as a technology forecast. We are working to 
develop a systematic system to help do technology assessment, in which the 3% stage (to finish 
pre-PICMET) targets policy action analysis. Combining the empirically-based work, which we are good 
at, we emphasize interactions with experts and publics via internet modes in this study. 


In this paper, we focus on our early attempts to build a systematic system for technology assessment. 
Our goal is to identify and assess the unintended, indirect, and delayed impacts through this system. 
This approach combines quantitative and qualitative analyses. This BDA analysis was a small-scale 
experiment. By inviting people to join the impact analysis discussion, such work could bolster 
development of technology itself. This type of information interchange could actively contribute to 
that development by helping to coalesce visions of innovation targets, to identify obstacles to be 
overcome and assets upon which to draw, and to perform impact assessment to identify potential 
beneficial and harmful effects. 
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